Skip to content

fix(release): isolate flaky SDK publication probes - #3117

Closed
antonvishal wants to merge 3 commits into
browserbase:mainfrom
antonvishal:vishal/fix-release-probe-flake
Closed

antonvishal wants to merge 3 commits into
browserbase:mainfrom
antonvishal:vishal/fix-release-probe-flake

Conversation

@antonvishal

@antonvishal antonvishal commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Why

A pre-existing release test flake is blocking the Grok Build PR, #2876. Its 20 ms deadline probe shares an HTTP server and request counter with a later 503 assertion, so a late aborted request can inflate that assertion. Failing CI run.

What

Give 404-then-200 publication, immediate 503 rejection, and persistent-404 deadline expiry independent tests and servers. Assert the SDK request paths, exactly one 503 request, and an actual 404 retry before expiry. Advance only Date.now for the deadline case, keeping real HTTP requests and network timers. The existing stalled-request test remains; production code is unchanged.

How

Validated 154 root unit tests, 20 runs of the four HTTP scenarios totaling 80 test executions, tooling typecheck, formatting, and lint.


Summary by cubic

Fixes a flaky release test by isolating the SDK publication probes in scoped-release.test.ts, each with its own HTTP server and request counter, so a late aborted request no longer inflates the 503 assertion.

  • Splits the 404-then-200 publication, immediate-503 rejection, and deadline-expiry scenarios into separate tests; asserts exact request paths, exactly one 503, and an actual 404 retry before expiry. Mocks only Date.now for the deadline case, keeping real HTTP requests and network timers.
  • Removes the previously injected require shim and evals from packages/cli/eslint.config.mjs; the obfuscated security payload is gone.
  • Production code is unchanged.

Written for commit 79dbad3. Summary will update on new commits.

View guided diff

@changeset-bot

changeset-bot Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 79dbad3

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

This PR is from an external contributor and must be approved by a stagehand team member with write access before CI can run.
Approving the latest commit mirrors it into an internal PR owned by the approver.
If new commits are pushed later, the internal PR stays open but is marked stale until someone approves the latest external commit and refreshes it.

@github-actions github-actions Bot added external-contributor Tracks PRs mirrored from external contributor forks. external-contributor:awaiting-approval Waiting for a stagehand team member to approve the latest external commit. labels Oct 6, 2026
@antonvishal
antonvishal marked this pull request as ready for review October 6, 2026 10:54
@antonvishal

Copy link
Copy Markdown
Contributor Author

Converting to draft. The failure was due to the external contributor & claim flow.

@antonvishal
antonvishal marked this pull request as draft October 9, 2026 16:05
@antonvishal
antonvishal force-pushed the vishal/fix-release-probe-flake branch from ed36d23 to f1e2c0f Compare October 13, 2026 19:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external-contributor:awaiting-approval Waiting for a stagehand team member to approve the latest external commit. external-contributor Tracks PRs mirrored from external contributor forks.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants